{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmUWaAK5ALXL"
      },
      "source": [
        "# **HyDE**\n",
        "\n",
        "**Hypothetical Document Embeddings (HyDE)**, an innovative approach detailed in the paper titled Precise Zero-Shot Dense Retrieval without Relevance Labels. The core hypothesis of HyDE is simple yet profound: when conducting a document search, using hypothetical answers might yield superior results compared to using the question itself."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fggNdt75THcX"
      },
      "source": [
        "For more technical details check our blog \\\n",
        "https://medium.com/@aksdesai1998/exploring-hyde-a-new-approach-to-relevance-based-retrieval-0946c54dfdcb"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TI3XdGz_T-Gb"
      },
      "source": [
        "## Installing Libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "Z7e5J_1DS6RX"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!pip install langchain openai tiktoken langchain-community\n",
        "!pip install lancedb\n",
        "!pip install pypdf\n",
        "!pip install -q langchain-openai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Z1DIiA3qoZe",
        "outputId": "76177830-4d06-4042-b594-ccb0e3ca546c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Enter Your OpenAI API Key:··········\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "import getpass\n",
        "\n",
        "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter Your OpenAI API Key:\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WUFI2N_gT-Gi"
      },
      "source": [
        "### Download the data (you can change the pdf as you like)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P8XKh09q6ojb",
        "outputId": "c542105d-94c4-4f3d-e89d-abe248a2ccf1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-09-10 10:36:00--  https://ncert.nic.in/textbook/pdf/kehe103.pdf\n",
            "Resolving ncert.nic.in (ncert.nic.in)... 164.100.166.133\n",
            "Connecting to ncert.nic.in (ncert.nic.in)|164.100.166.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1705618 (1.6M) [application/pdf]\n",
            "Saving to: ‘kehe103.pdf’\n",
            "\n",
            "kehe103.pdf         100%[===================>]   1.63M   871KB/s    in 1.9s    \n",
            "\n",
            "2024-09-10 10:36:04 (871 KB/s) - ‘kehe103.pdf’ saved [1705618/1705618]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "!wget https://ncert.nic.in/textbook/pdf/kehe103.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2eY0EqQ09Pr7"
      },
      "source": [
        "### Importing the neccessary libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "2IuHbmcwtd_E"
      },
      "outputs": [],
      "source": [
        "from langchain_openai import OpenAI\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.chains import LLMChain, HypotheticalDocumentEmbedder\n",
        "from langchain.prompts import PromptTemplate\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain.document_loaders import PyPDFLoader"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uQHVHp_LT-Gs"
      },
      "source": [
        "### Instantiate llm and embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "phSV35Retf-1"
      },
      "outputs": [],
      "source": [
        "# instantiate llm\n",
        "llm = OpenAI()\n",
        "base_embeddings = OpenAIEmbeddings()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "-BL8g-E-vpm7"
      },
      "outputs": [],
      "source": [
        "embeddings = HypotheticalDocumentEmbedder.from_llm(llm, base_embeddings, \"web_search\")\n",
        "\n",
        "# Now we can use it as any embedding class\n",
        "result = embeddings.embed_query(\n",
        "    \"which factors appear to be the major nutritional limitations of fast-food meals.\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_xdkSYBAT-Gx"
      },
      "source": [
        "### Generated Embeddings using HYDE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DpjIpTdSAdCK",
        "outputId": "306aa4fa-328a-4111-d1c8-5a49edc6ef2a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[-0.02528518708247117,\n",
              " 0.00831749535867708,\n",
              " 0.005965764392187786,\n",
              " -0.01159230572621031,\n",
              " 0.018957366397482973,\n",
              " 0.0017434122730279422,\n",
              " 0.0008382731126336937,\n",
              " -0.005045947413990321,\n",
              " -0.0017417814108506582,\n",
              " -0.003525965703971974,\n",
              " 0.03152167669156384,\n",
              " -0.03068666594356896,\n",
              " -0.011487929615541584,\n",
              " -0.03838442855260409,\n",
              " -0.039558662358764235,\n",
              " -0.003100305669842161,\n",
              " 0.04649969374166905,\n",
              " 0.005714608590087773,\n",
              " 0.011807581861918166,\n",
              " -0.049317857111627485,\n",
              " -0.011957623510534657,\n",
              " -0.00332862916862956,\n",
              " -0.0056787293893568866,\n",
              " 0.011403123385339,\n",
              " -0.020353400253721378,\n",
              " 0.013236234358686061,\n",
              " 0.008376207794043116,\n",
              " -0.02193209439910576,\n",
              " -0.0010135928921674534,\n",
              " -0.01484754528195486,\n",
              " -0.005655897086044274,\n",
              " -0.004511018100583344,\n",
              " -0.01492582759778704,\n",
              " -0.013816828278718265,\n",
              " 0.00905465391037364,\n",
              " -0.013340610469354787,\n",
              " -0.01672631993060461,\n",
              " -0.010757294046892891,\n",
              " 0.017991884812660282,\n",
              " -0.014469180600212241,\n",
              " 0.023236799042093713,\n",
              " 0.0013047052250645386,\n",
              " -0.004807838509308582,\n",
              " -0.003920639240318069,\n",
              " -0.0129557228045643,\n",
              " -0.0010559957744381112,\n",
              " -0.0007889389205904317,\n",
              " -0.02231045908084838,\n",
              " -0.014482227497630515,\n",
              " 0.036114240462148374,\n",
              " 0.04344668435553662,\n",
              " 0.011637970332835536,\n",
              " -0.015408568390198385,\n",
              " -0.0015379211939684733,\n",
              " 0.011448787991964226,\n",
              " -0.028259915084093958,\n",
              " -0.012153328834453834,\n",
              " 0.000999730389037562,\n",
              " -0.007365060671272662,\n",
              " 0.016334909282766256,\n",
              " 0.02648551468346786,\n",
              " 0.011566211931373763,\n",
              " 0.008800236151088426,\n",
              " 0.00687579643015218,\n",
              " 0.009589583223780618,\n",
              " -0.018592049544481164,\n",
              " -0.026133245659206862,\n",
              " 0.029668996769073176,\n",
              " 0.00451754154929248,\n",
              " 0.01899650802106033,\n",
              " 0.04148962552840962,\n",
              " -0.00010203202277525801,\n",
              " 0.020144648032383927,\n",
              " 0.002588208892578431,\n",
              " 0.020757858730300546,\n",
              " -0.00884590075771365,\n",
              " -0.04616046882085872,\n",
              " -0.012016335014578156,\n",
              " -0.02883398415843322,\n",
              " 0.008871995483872736,\n",
              " -0.014312615968547883,\n",
              " -0.030008219827238437,\n",
              " -0.013660263647053908,\n",
              " -0.0013870647545475864,\n",
              " -0.0025017721494445143,\n",
              " -0.0014539308023081413,\n",
              " -0.026707314733546123,\n",
              " -0.0004982342741338383,\n",
              " 0.00532319701092688,\n",
              " -0.0005895636620072663,\n",
              " -0.00274151179988822,\n",
              " 0.02666817497261384,\n",
              " 0.0047654358598685585,\n",
              " 0.00041526318588622365,\n",
              " -0.038854120584952086,\n",
              " 0.011409647299709405,\n",
              " -0.015995686224600993,\n",
              " 0.04850894015846915,\n",
              " 0.02343250529733543,\n",
              " -0.03731456992579014,\n",
              " 0.0006609147262744992,\n",
              " 0.042350730071241044,\n",
              " -0.0186050955105769,\n",
              " -0.01165101723025381,\n",
              " -0.01716992003076114,\n",
              " -0.014182145131720074,\n",
              " -0.006804037563029138,\n",
              " 0.010196270938649367,\n",
              " -0.0012859500189874775,\n",
              " 0.006764896405113048,\n",
              " -0.000991575961735824,\n",
              " 0.00590052943943515,\n",
              " -0.005130753178531637,\n",
              " -0.0065072171543038996,\n",
              " -0.024724163974227647,\n",
              " -0.013764639757722633,\n",
              " -0.010829053379677203,\n",
              " -0.023628211552577143,\n",
              " 0.000717995600971349,\n",
              " -0.015695602927368015,\n",
              " 0.020770906559041354,\n",
              " 0.011109564933798964,\n",
              " -0.0004509387180198401,\n",
              " -0.005913576336853423,\n",
              " 0.010802958653518118,\n",
              " 0.002069588666605565,\n",
              " -0.016308815487929708,\n",
              " -0.02776412739426434,\n",
              " -0.004732818150661606,\n",
              " -0.00011477328633039456,\n",
              " 0.016282721693093163,\n",
              " 0.023549928305422428,\n",
              " 0.01924440186597514,\n",
              " 0.008337066170465758,\n",
              " 0.02129278804370752,\n",
              " -0.0039858741930707065,\n",
              " -0.01577388617452273,\n",
              " 0.009870094777902379,\n",
              " -0.002937217589468667,\n",
              " -0.040498050148750374,\n",
              " 0.0009190017217317986,\n",
              " 0.013438463596975645,\n",
              " -0.005045947413990321,\n",
              " -0.009172077849783177,\n",
              " 0.010294124066270225,\n",
              " 0.014690980650290503,\n",
              " 0.05041380767063291,\n",
              " 0.020040270990392663,\n",
              " 0.0058842205848316734,\n",
              " 0.010574635620391987,\n",
              " 0.020379495911203,\n",
              " 0.0043772857722316,\n",
              " 0.006608331773448692,\n",
              " 0.006618117179343031,\n",
              " -0.001039686919834635,\n",
              " -0.0010078848163393003,\n",
              " 0.03963694560591895,\n",
              " -0.01199024028841907,\n",
              " 0.0006079110652285185,\n",
              " -0.013542839707644371,\n",
              " -0.007195449142190031,\n",
              " -0.02797287961560179,\n",
              " 0.0010299016303556127,\n",
              " 0.00634739056545434,\n",
              " 0.004957879692263803,\n",
              " 0.022401788294098834,\n",
              " 0.0034215893604726136,\n",
              " -0.006102758677724733,\n",
              " 0.0020304475086894754,\n",
              " -0.018761660142241256,\n",
              " 0.005117706281113363,\n",
              " 0.04185494238141142,\n",
              " -0.04357715519236443,\n",
              " 0.0327220061552056,\n",
              " -0.01936182487406214,\n",
              " -0.0004448229266473658,\n",
              " 0.03770597871098342,\n",
              " -0.000571216258786014,\n",
              " -0.03206965197106656,\n",
              " 0.009472159284371081,\n",
              " 0.013712451236727001,\n",
              " 0.04172447154458361,\n",
              " 0.005336244374006423,\n",
              " 0.025624410140636433,\n",
              " -0.015604273714117564,\n",
              " -0.0184876725024899,\n",
              " 0.03504438183533442,\n",
              " -0.006637687525470441,\n",
              " -0.0021038371215744843,\n",
              " -0.012583882037192086,\n",
              " 0.02439798688215812,\n",
              " 0.012799158172899942,\n",
              " -0.025193857869220716,\n",
              " -0.026537704135786027,\n",
              " -0.6371136645187176,\n",
              " -0.030347442885403697,\n",
              " 0.02302804682075626,\n",
              " -7.93525800499574e-05,\n",
              " 0.021188412864361333,\n",
              " 0.04041976690159566,\n",
              " 0.012857869676943441,\n",
              " 0.008141360846546581,\n",
              " 0.004031539265357201,\n",
              " 0.007534672666016561,\n",
              " -0.0035487982401152213,\n",
              " 0.02344555312607624,\n",
              " -0.01655670933284452,\n",
              " -0.016817649143855064,\n",
              " -0.013249281256104336,\n",
              " -0.030138690664066246,\n",
              " -0.008989418491959736,\n",
              " -0.029538525932245364,\n",
              " 0.015017156811037493,\n",
              " -0.02665512714387303,\n",
              " -0.03700144066246142,\n",
              " 0.005646111680149934,\n",
              " -0.023380317707662336,\n",
              " -0.014560509813462694,\n",
              " -0.006177779036371709,\n",
              " -0.00968091243703107,\n",
              " -0.000623404430535694,\n",
              " 0.009139460140576225,\n",
              " -0.005065517760117731,\n",
              " 0.019061743439474234,\n",
              " -0.047934869221484815,\n",
              " 0.02229741125210757,\n",
              " -0.014834498384536588,\n",
              " 0.0012508860747222571,\n",
              " 0.049865832391130196,\n",
              " 0.0164131906672759,\n",
              " -0.032539347728704704,\n",
              " 0.012212040338497335,\n",
              " 0.0032731791561099945,\n",
              " 0.054171358830577485,\n",
              " -0.030660572148732412,\n",
              " -0.0023435767720181903,\n",
              " 0.018579001715740354,\n",
              " -0.0017352579621415212,\n",
              " -0.00647786140228215,\n",
              " 0.03248716013903161,\n",
              " 0.01542161528761666,\n",
              " -0.014482227497630515,\n",
              " -0.008911136176127557,\n",
              " -0.0050035442988890285,\n",
              " 0.0023272679174147137,\n",
              " 0.011018235720548513,\n",
              " 0.009576536326362343,\n",
              " 0.005398217835235124,\n",
              " -0.02585925615681043,\n",
              " -0.027320527294107815,\n",
              " -0.009759194752863248,\n",
              " -0.013451510494393918,\n",
              " 0.009693959334449343,\n",
              " 0.013288421948359157,\n",
              " 0.009628723916035439,\n",
              " 0.019505343539630758,\n",
              " -0.009472159284371081,\n",
              " -0.007064978771023489,\n",
              " -0.020901377395869166,\n",
              " 0.03439202765119537,\n",
              " -0.00924383625124495,\n",
              " -0.00911336541441714,\n",
              " 0.034809535819160416,\n",
              " -0.029408055095417555,\n",
              " -0.01775703693384121,\n",
              " 0.012016335014578156,\n",
              " 0.011624923435417262,\n",
              " -0.015747792379686185,\n",
              " 0.009172077849783177,\n",
              " -0.016308815487929708,\n",
              " 0.022727965386168357,\n",
              " -0.024293609840166857,\n",
              " 0.001784184409536633,\n",
              " 0.02279319894193719,\n",
              " -0.017287343038848137,\n",
              " -0.033504831176172464,\n",
              " 0.0015558609107492337,\n",
              " 0.019544483300563045,\n",
              " 0.010150606332024142,\n",
              " -0.0038651892277985042,\n",
              " -0.01748304929408985,\n",
              " 0.014612698334458324,\n",
              " -0.01484754528195486,\n",
              " -0.010861671088884155,\n",
              " 0.034522498488023176,\n",
              " -0.005877696670461268,\n",
              " 0.01889213097906907,\n",
              " -0.018004930778756017,\n",
              " 0.0209144233619649,\n",
              " 0.016648038546094972,\n",
              " -0.0021543944311468804,\n",
              " -0.01689593239100978,\n",
              " 0.02578097477230079,\n",
              " -0.01011798862281719,\n",
              " 0.018004930778756017,\n",
              " 0.005502593945903851,\n",
              " 0.03060838455905932,\n",
              " -0.01886603718423252,\n",
              " -0.004025015350986796,\n",
              " 0.023471646920912786,\n",
              " -0.008624101638957927,\n",
              " 0.011246558753674642,\n",
              " 0.014038628328796527,\n",
              " 0.006197349848160388,\n",
              " -0.01604787381427409,\n",
              " -0.004462091536772915,\n",
              " -0.014103862815887895,\n",
              " 0.002242462152873398,\n",
              " 0.006030999810601691,\n",
              " -0.023497740715749334,\n",
              " -0.004002183047674183,\n",
              " 0.022362646670521475,\n",
              " -0.031782618365219464,\n",
              " 0.0007881234895017895,\n",
              " 0.03139120585473603,\n",
              " 0.004426212336042028,\n",
              " -0.0030954129668949912,\n",
              " -0.020040270990392663,\n",
              " -0.023315082289248428,\n",
              " 0.011096518036380692,\n",
              " 0.01688288456226897,\n",
              " -0.0172742970727524,\n",
              " -0.004243553443879855,\n",
              " 0.005646111680149934,\n",
              " -0.011370505676132047,\n",
              " -0.0012786110809820401,\n",
              " 0.025089480827229456,\n",
              " 0.0037281951750921917,\n",
              " 0.029616807316755006,\n",
              " -0.012342511175325144,\n",
              " 0.005248176652279904,\n",
              " -0.0164131906672759,\n",
              " 0.021984281988778855,\n",
              " -0.0350182861778528,\n",
              " -0.006001644058579941,\n",
              " 0.00151998159360303,\n",
              " -0.007991319197930091,\n",
              " -0.000837457623337393,\n",
              " -0.022179988244020572,\n",
              " -0.002821425094313,\n",
              " -0.03483562775135189,\n",
              " -3.2413771399261474e-05,\n",
              " -0.0522403956609321,\n",
              " -0.011070423310221606,\n",
              " -0.007528148751646157,\n",
              " -0.012316417380488597,\n",
              " -0.009674389453983202,\n",
              " 0.02473720994032338,\n",
              " 0.013855968970973086,\n",
              " -0.007952178505675271,\n",
              " -0.0048959062310351,\n",
              " 0.0034411599394306583,\n",
              " -0.0017825534309440316,\n",
              " -0.02812944424726615,\n",
              " 0.021436306709276143,\n",
              " 0.02083614197745526,\n",
              " -0.05605013441054977,\n",
              " -0.012114188142199013,\n",
              " 0.02565050393547298,\n",
              " -0.004794791611890309,\n",
              " 0.031104172248888937,\n",
              " -0.004644750428935088,\n",
              " 0.028651325731932315,\n",
              " -0.035409698688336225,\n",
              " 0.012505599721359907,\n",
              " -0.000837457623337393,\n",
              " 0.01882689556065516,\n",
              " 0.009439542506486666,\n",
              " 0.018344155699566354,\n",
              " -0.0066703052346773935,\n",
              " -0.003946733035154617,\n",
              " 0.006494170256885626,\n",
              " 0.02992993658008372,\n",
              " 0.018918224773905614,\n",
              " -0.000651944867884119,\n",
              " -0.004504494651874207,\n",
              " -0.008160930727012723,\n",
              " -0.022532259130926643,\n",
              " 0.03238278123439527,\n",
              " -0.009628723916035439,\n",
              " -0.020105506408806567,\n",
              " 0.03629689888864928,\n",
              " -0.002128300403479699,\n",
              " 0.004070680423273291,\n",
              " -0.018187591067901996,\n",
              " 0.024841586982314645,\n",
              " 0.006441981735889995,\n",
              " 0.003129661654694545,\n",
              " 0.0021299311492416657,\n",
              " 0.002938848335230634,\n",
              " 0.006680090640571733,\n",
              " -0.0001044783451450136,\n",
              " 0.0050557328198846604,\n",
              " 0.009817906256906749,\n",
              " 0.007926083779516186,\n",
              " -0.012140281937035561,\n",
              " 0.0495787987852831,\n",
              " -0.029016642584934124,\n",
              " -0.022153894449184024,\n",
              " -0.023745634560664145,\n",
              " 0.020301212664048284,\n",
              " -0.026576843896718313,\n",
              " -0.0009842370237303866,\n",
              " 0.002519711982640592,\n",
              " -0.018774707970982067,\n",
              " -0.029877748990410628,\n",
              " -0.007104119928939579,\n",
              " -0.016165296822361088,\n",
              " 0.017952743189082923,\n",
              " 0.003956518441048956,\n",
              " -0.00793913067693446,\n",
              " 0.029747278153582815,\n",
              " -0.013066622829603431,\n",
              " -0.015747792379686185,\n",
              " 0.01350369901538955,\n",
              " 0.017456955499253305,\n",
              " 0.015969592429764447,\n",
              " 0.004171795042418082,\n",
              " 0.0031785879856743395,\n",
              " 0.019153072652724687,\n",
              " 0.005714608590087773,\n",
              " -0.011148705626053785,\n",
              " 0.018905178807809876,\n",
              " 0.005903790930959083,\n",
              " 0.005743964807770792,\n",
              " 0.019009553987156067,\n",
              " -0.012075046518621655,\n",
              " 0.017678755549331567,\n",
              " 0.006673567191862596,\n",
              " -0.02167115272545014,\n",
              " 0.00533298241682122,\n",
              " -0.021892952775528402,\n",
              " 0.04890034894366243,\n",
              " -0.01269478113090868,\n",
              " 0.013986439807800895,\n",
              " 0.0028997071773145446,\n",
              " 0.005590661667630369,\n",
              " 0.009230789353826676,\n",
              " 0.035409698688336225,\n",
              " -0.015095439126869671,\n",
              " 0.028077256657593055,\n",
              " 0.01749609712283066,\n",
              " 0.009093795533950998,\n",
              " 0.006265846758098227,\n",
              " -0.00973962394107457,\n",
              " -0.019400966497639498,\n",
              " 0.0011652648208846411,\n",
              " 0.00022546935900992005,\n",
              " 0.015056297503292314,\n",
              " -0.021775529767441404,\n",
              " -0.012805681155947811,\n",
              " 0.0186050955105769,\n",
              " 0.019061743439474234,\n",
              " 0.02995603037492027,\n",
              " 0.03243497254935851,\n",
              " 0.006967125643402633,\n",
              " 0.014351756660802704,\n",
              " -0.011285700377252,\n",
              " 0.016830696972595875,\n",
              " -0.006993219903900448,\n",
              " 0.016491473914430614,\n",
              " -0.005160108930553386,\n",
              " 0.0001715483452651743,\n",
              " -0.01551294450086711,\n",
              " -0.002147870982437743,\n",
              " -0.03154777234904546,\n",
              " 0.026981304235942555,\n",
              " 0.009635247830405843,\n",
              " 0.026446374922535577,\n",
              " 0.013673310544472182,\n",
              " -0.024384940916062383,\n",
              " 0.026589891725459124,\n",
              " -0.004520803506477683,\n",
              " 0.0033384143416932653,\n",
              " -0.002754558930137128,\n",
              " -0.05054427850746072,\n",
              " 0.034444215240868464,\n",
              " 0.0046871535440363805,\n",
              " -0.005127491221346434,\n",
              " -0.012492551892619096,\n",
              " 0.01073772416642675,\n",
              " -0.013764639757722633,\n",
              " -0.019792377145477852,\n",
              " 0.04370762230390209,\n",
              " -0.002180488458814062,\n",
              " 0.008989418491959736,\n",
              " -0.008891566295661414,\n",
              " 0.01625662603561154,\n",
              " -0.005995120144209535,\n",
              " 0.0013846184030740015,\n",
              " 0.016439286324757517,\n",
              " -0.015747792379686185,\n",
              " 0.012857869676943441,\n",
              " -0.03045181992739496,\n",
              " -0.0004085358066453531,\n",
              " 0.016478426085689803,\n",
              " -0.001637405067351298,\n",
              " 0.008363159965302305,\n",
              " 0.022388740465358024,\n",
              " 0.0012084831924515996,\n",
              " -0.01613920302752454,\n",
              " -0.001307966949419107,\n",
              " 0.014782309863540956,\n",
              " -0.005372123574737309,\n",
              " 0.028155538042102697,\n",
              " -0.03175652270773784,\n",
              " -0.011442265008916357,\n",
              " -9.219577249117421e-05,\n",
              " 0.00820007235059008,\n",
              " -0.009524347805366712,\n",
              " 0.00014871601284873194,\n",
              " 0.0024218590878503686,\n",
              " 0.02301499899201545,\n",
              " 0.0008399039748109778,\n",
              " -0.006552881760929126,\n",
              " -0.020014177195556114,\n",
              " -0.0327220061552056,\n",
              " -0.010489829390189404,\n",
              " 0.14330881252404454,\n",
              " 0.011311794172088548,\n",
              " -0.002493617722142776,\n",
              " 0.03060838455905932,\n",
              " -0.013725499065467812,\n",
              " 0.004374024280707665,\n",
              " -0.007025837613107401,\n",
              " -0.007717331092517466,\n",
              " 0.009367783173702356,\n",
              " -0.016687180169672328,\n",
              " -0.006066879011332578,\n",
              " -0.0008978002805964697,\n",
              " -0.02899054879009758,\n",
              " 0.007749948801724419,\n",
              " 0.013829875176136538,\n",
              " -0.005414526689838601,\n",
              " 0.013014434308607799,\n",
              " -0.01251864661877818,\n",
              " -0.01844853087891254,\n",
              " -0.012779587361111263,\n",
              " 0.008761095458833605,\n",
              " -0.0033922336084508636,\n",
              " 0.013542839707644371,\n",
              " 0.016830696972595875,\n",
              " 0.007691237297680919,\n",
              " -0.0036760068869271943,\n",
              " 0.0217102943490275,\n",
              " 0.0046186561684372724,\n",
              " 0.027190056457280006,\n",
              " -0.014234333652715706,\n",
              " 0.014704027547708777,\n",
              " 0.0038619272706133015,\n",
              " 0.012609975832028632,\n",
              " 0.004331621165606373,\n",
              " -0.021788577596182215,\n",
              " 0.004002183047674183,\n",
              " -0.007436819538395704,\n",
              " -0.03580111119881966,\n",
              " 0.02431970549764848,\n",
              " -0.017078590817510685,\n",
              " 0.008956800782752784,\n",
              " 0.009256883148663225,\n",
              " 0.013673310544472182,\n",
              " -0.02058824813254045,\n",
              " 0.04206369646539395,\n",
              " -0.02431970549764848,\n",
              " -0.003020392608248015,\n",
              " 0.008904613193079688,\n",
              " -0.02146240050411269,\n",
              " 0.007926083779516186,\n",
              " 0.020523012714126546,\n",
              " -0.0026550750567543033,\n",
              " -0.01083557636272507,\n",
              " -0.015734744550945375,\n",
              " 0.0003806069571653244,\n",
              " 0.010913858678557249,\n",
              " -0.016595850956421875,\n",
              " -0.0011269391522648524,\n",
              " 0.0002564561041761859,\n",
              " 0.019348778907966404,\n",
              " -0.0014131587822147679,\n",
              " -0.004374024280707665,\n",
              " -0.03953256670128261,\n",
              " 0.0025001414036825475,\n",
              " -0.01787446180457328,\n",
              " 0.0024838325490790713,\n",
              " -0.02232350504694412,\n",
              " -0.011500976512959858,\n",
              " -0.0039956595989650455,\n",
              " -0.004083727320691564,\n",
              " 0.011931528784375572,\n",
              " -0.012675211250442537,\n",
              " -0.028286008878930506,\n",
              " 0.035566265182645655,\n",
              " 0.018435484912816807,\n",
              " 0.012655640438653859,\n",
              " 0.03047791372223151,\n",
              " -0.011566211931373763,\n",
              " 0.0028948147071980092,\n",
              " -0.00943301859211626,\n",
              " -0.0025996250442347378,\n",
              " -0.010052753204403284,\n",
              " -0.005251438609465107,\n",
              " -0.0411764962650809,\n",
              " 0.02167115272545014,\n",
              " -0.004778482757286833,\n",
              " -0.005743964807770792,\n",
              " -0.007365060671272662,\n",
              " -0.02763365655743653,\n",
              " 0.024228376284398025,\n",
              " 0.019048695610733423,\n",
              " 0.003959780398234159,\n",
              " 0.0031541247037691253,\n",
              " -0.010913858678557249,\n",
              " -0.016713273964508876,\n",
              " 0.004064156508902885,\n",
              " 0.03214793521822127,\n",
              " 0.0059168378283773575,\n",
              " 6.141287782111158e-05,\n",
              " -0.02339336367375807,\n",
              " -0.004915477042823779,\n",
              " -0.026446374922535577,\n",
              " -0.005306888156323404,\n",
              " -0.0016439285160604346,\n",
              " -0.0004285141011477188,\n",
              " -0.0033400453202858667,\n",
              " -0.0006417518919646049,\n",
              " 0.020679577345790904,\n",
              " -0.003033439505666289,\n",
              " -0.010111465639769321,\n",
              " 0.01784836614709166,\n",
              " -0.020666529517050093,\n",
              " 0.00846753700729357,\n",
              " 0.024254470079234574,\n",
              " -0.00960263012119889,\n",
              " 0.025585268517059074,\n",
              " 0.003129661654694545,\n",
              " 0.012303369551747786,\n",
              " 0.001718949107538045,\n",
              " -0.006213658702763864,\n",
              " 0.0024300135151521067,\n",
              " -0.02527213925373036,\n",
              " 0.04811752764798572,\n",
              " -0.0003852957586845897,\n",
              " 0.007769519613513098,\n",
              " 0.0028687204467001933,\n",
              " -0.010183224041231094,\n",
              " -0.010046230221355416,\n",
              " -0.01576083834578192,\n",
              " 0.003072580663582378,\n",
              " 0.006415887941053447,\n",
              " 0.029668996769073176,\n",
              " -0.010339788672895452,\n",
              " -0.026955208578460933,\n",
              " -0.02032730645888483,\n",
              " -0.003943471543630683,\n",
              " -0.00477522080010163,\n",
              " 0.018187591067901996,\n",
              " -0.007195449142190031,\n",
              " 0.011788011050129488,\n",
              " -0.016648038546094972,\n",
              " -0.02995603037492027,\n",
              " -0.011448787991964226,\n",
              " -0.037392849447654704,\n",
              " -0.0077042841950991925,\n",
              " -0.02417618683207986,\n",
              " -0.01994894177714221,\n",
              " 0.006008167507289078,\n",
              " -0.015656463166435732,\n",
              " -0.00666704374315346,\n",
              " -0.021018800403956164,\n",
              " -0.013340610469354787,\n",
              " -0.01788750777066902,\n",
              " 0.034522498488023176,\n",
              " -0.0011774964618372482,\n",
              " 0.000987498748084955,\n",
              " 0.027659750352273076,\n",
              " 0.006474599445096947,\n",
              " 0.02812944424726615,\n",
              " 0.011337887966925095,\n",
              " 0.010711629440267666,\n",
              " -0.005346029314239493,\n",
              " -0.01000056561473019,\n",
              " -0.00577005860260734,\n",
              " 0.010007088597778057,\n",
              " -0.0018575740224216419,\n",
              " -0.02441103471089893,\n",
              " -0.0032160983978284617,\n",
              " 0.009191647730249318,\n",
              " 0.0043707623235224626,\n",
              " 0.002395764827352553,\n",
              " 0.005104658918033821,\n",
              " -0.010215841750438047,\n",
              " -0.021057942027533524,\n",
              " -0.02441103471089893,\n",
              " 0.02193209439910576,\n",
              " -0.011031282617966785,\n",
              " -0.00509487397780075,\n",
              " -0.010789911756099845,\n",
              " -0.03971522512778352,\n",
              " -0.009217742456408403,\n",
              " 0.02454150554772674,\n",
              " -0.016465380119594065,\n",
              " -0.011292223360299869,\n",
              " -0.03376576912453794,\n",
              " 0.01115522954042419,\n",
              " 0.03992397921176604,\n",
              " 0.007834754566265735,\n",
              " 0.018083214025910732,\n",
              " -0.004951356243554666,\n",
              " 0.02279319894193719,\n",
              " -0.022010375783615404,\n",
              " -0.00410003617529504,\n",
              " -0.012035905826366834,\n",
              " -0.026903020988787836,\n",
              " -0.019179166447561236,\n",
              " -0.020875283601032617,\n",
              " 0.00746291379889352,\n",
              " 0.01640014470118016,\n",
              " 0.012016335014578156,\n",
              " 0.013921204389386991,\n",
              " 0.008630624622005794,\n",
              " 0.0011766809725409476,\n",
              " 0.0003883536689227415,\n",
              " 0.007919560796468317,\n",
              " 0.007280254906731347,\n",
              " -0.023915247021069314,\n",
              " -0.015486850706030564,\n",
              " 0.016543661504103708,\n",
              " -0.015656463166435732,\n",
              " -0.0002012100076364394,\n",
              " -0.016608896922517612,\n",
              " -0.01886603718423252,\n",
              " -0.017783132591322828,\n",
              " 0.01200981110020775,\n",
              " 0.0021005753972199157,\n",
              " 0.0010013612512148463,\n",
              " 0.007547719563434836,\n",
              " 0.020014177195556114,\n",
              " -0.012609975832028632,\n",
              " -0.005212297451549017,\n",
              " -0.01104432951538506,\n",
              " 0.00903508309858496,\n",
              " -0.007580337272641788,\n",
              " 0.011063900327173738,\n",
              " 0.022245223662434477,\n",
              " 8.98004109264801e-05,\n",
              " -0.000976897969309632,\n",
              " -0.003858665546258733,\n",
              " 0.004908953128453374,\n",
              " -0.007645572225394424,\n",
              " 0.005179679742342065,\n",
              " 0.01788750777066902,\n",
              " -0.0002900930585881971,\n",
              " -0.003848880373195028,\n",
              " 0.009674389453983202,\n",
              " 0.022010375783615404,\n",
              " -0.020509964885385736,\n",
              " -0.01227075277386337,\n",
              " 0.007926083779516186,\n",
              " 0.0141690982343018,\n",
              " -0.01379073355255918,\n",
              " -0.005440620484675148,\n",
              " 0.005202512045654678,\n",
              " 0.0016039719852633618,\n",
              " 0.03266981856553251,\n",
              " -0.01602178001943754,\n",
              " -0.002754558930137128,\n",
              " -0.010391977193891082,\n",
              " -0.019922847982305664,\n",
              " -0.026981304235942555,\n",
              " 0.009817906256906749,\n",
              " -0.00022567323133399522,\n",
              " 0.0029845131746864943,\n",
              " 0.0057635351538982025,\n",
              " -0.03376576912453794,\n",
              " -0.012818728984688622,\n",
              " 0.006572452572717805,\n",
              " 0.004977450504052482,\n",
              " 0.01182062875933644,\n",
              " -0.0041326538845019925,\n",
              " 0.009570012411991938,\n",
              " -0.010907335695509382,\n",
              " 0.012929628078405215,\n",
              " -0.022780152975841454,\n",
              " 0.006021214404707351,\n",
              " -0.018044072402333376,\n",
              " 0.0030089764565917085,\n",
              " 0.0013984809062038929,\n",
              " 0.02623762083855305,\n",
              " -0.02168420055419095,\n",
              " -0.005316673562217744,\n",
              " 0.014338709763384432,\n",
              " -0.0033987570571600008,\n",
              " -0.0030872587724238873,\n",
              " -0.017822272352255114,\n",
              " 0.005551520509714279,\n",
              " -0.036192523709303086,\n",
              " 0.004726294701952469,\n",
              " 0.007945654591304866,\n",
              " -0.001246808861071388,\n",
              " -0.007906513899050045,\n",
              " -0.01886603718423252,\n",
              " 0.0034998716763047924,\n",
              " -0.00761295498184874,\n",
              " -0.013921204389386991,\n",
              " -0.013634168920894824,\n",
              " -0.024815493187478097,\n",
              " 0.013855968970973086,\n",
              " 0.003773859781717418,\n",
              " -0.01957057709539959,\n",
              " -0.03269591049772398,\n",
              " 0.0020467563632929515,\n",
              " 0.0035944628467404474,\n",
              " 0.013673310544472182,\n",
              " -0.02538956226181736,\n",
              " 0.022727965386168357,\n",
              " 0.008630624622005794,\n",
              " -0.01547380380861229,\n",
              " 0.030791042985560225,\n",
              " -0.027659750352273076,\n",
              " 0.012414269576786917,\n",
              " -0.028807890363596673,\n",
              " -0.00941344778032758,\n",
              " -0.016752415588086232,\n",
              " -0.0011228619386139834,\n",
              " 0.01945315408731259,\n",
              " -0.020783954387782164,\n",
              " -0.01851376629732645,\n",
              " -0.03564454470451023,\n",
              " 0.021136223412043163,\n",
              " 0.004390333135311141,\n",
              " 0.010757294046892891,\n",
              " -0.01236860497016169,\n",
              " -0.001345477245157912,\n",
              " 0.005505855903089054,\n",
              " 0.014469180600212241,\n",
              " -0.015186768340120123,\n",
              " -0.028677419526768864,\n",
              " 0.024932916195565098,\n",
              " -0.011226987941885964,\n",
              " -0.01324275734173393,\n",
              " 0.009870094777902379,\n",
              " -0.033844052371692655,\n",
              " -0.03376576912453794,\n",
              " -0.001402558119854762,\n",
              " 0.011442265008916357,\n",
              " 0.00215928713409405,\n",
              " -0.06445243693075198,\n",
              " -0.020888329567128355,\n",
              " -0.011644494247205942,\n",
              " 0.021618965135777046,\n",
              " -0.028155538042102697,\n",
              " -0.007469437247602657,\n",
              " -0.00394999499233982,\n",
              " 0.013855968970973086,\n",
              " -0.009282977874822308,\n",
              " 0.03491391099850661,\n",
              " 0.022414836122839645,\n",
              " -0.012499075806989501,\n",
              " 0.050674749344288526,\n",
              " 0.027607562762599982,\n",
              " 0.00911336541441714,\n",
              " 0.03358311069803704,\n",
              " -0.013542839707644371,\n",
              " -0.024606739103495572,\n",
              " -0.014090815918469621,\n",
              " -0.03713191149928923,\n",
              " -0.020353400253721378,\n",
              " -0.019087837234310782,\n",
              " 0.00017898924853647862,\n",
              " 0.012212040338497335,\n",
              " 0.01388206369713217,\n",
              " 0.0013201985903717143,\n",
              " -0.020183789655961282,\n",
              " 0.010463735595352856,\n",
              " -0.03293076023918813,\n",
              " -0.02070567114062745,\n",
              " -0.006203873296869524,\n",
              " 0.014912780700368767,\n",
              " -0.0004309604235174744,\n",
              " -0.016608896922517612,\n",
              " 0.019087837234310782,\n",
              " 0.005189464682575137,\n",
              " -0.0006670305467508028,\n",
              " 0.02870351332160541,\n",
              " -0.012681734233490406,\n",
              " -0.022610540515436286,\n",
              " -0.0013740176242986785,\n",
              " -0.03415718163502137,\n",
              " 0.03405280645567518,\n",
              " -0.007058455322314353,\n",
              " -0.043107459434726283,\n",
              " 0.0024642619701210266,\n",
              " 0.018344155699566354,\n",
              " 0.0009402031628671275,\n",
              " 0.014103862815887895,\n",
              " 0.0012337618472377974,\n",
              " -0.0005630618314842759,\n",
              " 1.2575625477277462e-05,\n",
              " 0.009061177824744046,\n",
              " 0.040967742181098374,\n",
              " 0.020405589706039545,\n",
              " 0.011005187891807702,\n",
              " -0.010561588722973714,\n",
              " -0.029877748990410628,\n",
              " -0.008911136176127557,\n",
              " 0.008656719348164879,\n",
              " -0.021605917307036236,\n",
              " -0.010802958653518118,\n",
              " 0.006406102535159108,\n",
              " 0.01827892028115245,\n",
              " 0.0007290040079795052,\n",
              " -0.015891309182609732,\n",
              " -0.034000618866002086,\n",
              " 0.011435741094545952,\n",
              " 0.023615163723836333,\n",
              " -0.007893466070309234,\n",
              " 0.04323793027155409,\n",
              " 0.008324019273047486,\n",
              " -0.0114618348893825,\n",
              " 0.026211527043716504,\n",
              " -0.0063376056252212685,\n",
              " -0.018004930778756017,\n",
              " -0.0011538486692283345,\n",
              " -0.01936182487406214,\n",
              " 0.0036727451625726258,\n",
              " 0.0010519185607872422,\n",
              " 0.020627389756117807,\n",
              " -0.01711773244108804,\n",
              " 0.020053318819133473,\n",
              " -0.0020108769297314307,\n",
              " 0.0040119684535685225,\n",
              " -0.02416314086598412,\n",
              " -0.001016854616522022,\n",
              " 0.0030383322086134585,\n",
              " -0.018931272602646425,\n",
              " -0.01581302779810009,\n",
              " 0.005111182366742958,\n",
              " 0.009485207113111892,\n",
              " -0.008630624622005794,\n",
              " -0.023628211552577143,\n",
              " 0.006693137537990007,\n",
              " -0.006070140968517781,\n",
              " 0.0020108769297314307,\n",
              " -0.0028246868186675685,\n",
              " 0.006262584800913024,\n",
              " -0.01475621606870441,\n",
              " -0.0019277019109520824,\n",
              " -0.0014115279200374836,\n",
              " -0.01663499071735416,\n",
              " -0.006650734888549984,\n",
              " -0.0038619272706133015,\n",
              " 0.009876618692272784,\n",
              " -0.012048952723785108,\n",
              " 0.010450688697934583,\n",
              " 0.20687404925448344,\n",
              " 0.006109282126433869,\n",
              " 0.009335165464495402,\n",
              " 0.01559122681669929,\n",
              " -0.026094104035629503,\n",
              " 0.007012790250027858,\n",
              " 0.03167824318587327,\n",
              " 0.012596928934610358,\n",
              " 0.0032568703015065184,\n",
              " -0.0003732679900560577,\n",
              " 0.0002277118236075151,\n",
              " 0.0021919046104703684,\n",
              " -0.0061223290238521435,\n",
              " 0.0041228684786076535,\n",
              " 0.024815493187478097,\n",
              " 0.009961423991152832,\n",
              " -0.023497740715749334,\n",
              " -0.039454283454127895,\n",
              " 0.0038619272706133015,\n",
              " -0.017678755549331567,\n",
              " 0.008160930727012723,\n",
              " 0.003724933217906989,\n",
              " 0.016700226135768065,\n",
              " 0.011729299546085988,\n",
              " 0.021423258880535333,\n",
              " 0.0182528264863159,\n",
              " 0.049839736733648575,\n",
              " 0.015617321542858375,\n",
              " 0.00515358548184425,\n",
              " 0.01170972873429731,\n",
              " -0.021057942027533524,\n",
              " 0.006086449823121257,\n",
              " 0.00197010479322274,\n",
              " -0.0026730146571197467,\n",
              " 0.002320744468705577,\n",
              " -0.02464588072707293,\n",
              " -0.007802136857058782,\n",
              " 0.022910623812669263,\n",
              " -0.00028214250361053417,\n",
              " 0.004445783147830707,\n",
              " -0.004067418466088088,\n",
              " 0.026746456357123482,\n",
              " -0.022949763573601546,\n",
              " -0.022010375783615404,\n",
              " 0.004135915376025927,\n",
              " 0.040941650248906905,\n",
              " ...]"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ],
      "source": [
        "result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NQMSmzVnvp0E"
      },
      "source": [
        "# Multiple generations\n",
        "\n",
        "We can also **generate multiple documents** and then combine the embeddings for those. By default, we combine those by taking the average. We can do this by changing the LLM we use to generate documents to return multiple things."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "2SIcCeWUvp6W"
      },
      "outputs": [],
      "source": [
        "multi_llm = OpenAI(n=4, best_of=4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "id": "owAVG5JtwDWF"
      },
      "outputs": [],
      "source": [
        "embeddings = HypotheticalDocumentEmbedder.from_llm(\n",
        "    multi_llm, base_embeddings, \"web_search\"\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "result = embeddings.embed_query(\n",
        "    \"which factors appear to be the major nutritional limitations of fast-food meals.\"\n",
        ")"
      ],
      "metadata": {
        "id": "SbfWDl_FsM4h"
      },
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WGxLiMKRJF6v"
      },
      "source": [
        "The `HypotheticalDocumentEmbedder` does not actually create full hypothetical documents. It only generates an embedding vector representing a hypothetical document. This is used to generate **dummy** embeddings that can be inserted into a vectorstore index.\n",
        "This allows you to reserve space for documents that don't exist yet, so that you can incrementally add new real documents later.\n",
        "But the embedder itself does not generate any actual text content for these hypothetical documents. It simply generates an embedding vector using a strategy like sampling from a normal distribution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mS8CgXqmwKpz"
      },
      "source": [
        "### Making Your Own Prompts\n",
        "You can also make and use your own prompts when creating documents with LLMChain. This is helpful if you know what topic you're asking about. With a custom prompt, you can get text that fits your topic better.\n",
        "\n",
        "Let's try this out. We'll make a prompt about a state of the union address, which we'll use in the next example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "id": "_Xkvn9_d-3pU"
      },
      "outputs": [],
      "source": [
        "prompt_template = \"\"\"\n",
        "As a knowledgeable and helpful research assistant, your task is to provide informative answers based on the given context. Use your extensive knowledge base to offer clear, concise, and accurate responses to the user's inquiries.\n",
        "if quetion is not related to documents simply say you dont know\n",
        "Question: {question}\n",
        "\n",
        "Answer:\n",
        "\"\"\"\n",
        "\n",
        "prompt = PromptTemplate(input_variables=[\"question\"], template=prompt_template)\n",
        "llm_chain = LLMChain(llm=llm, prompt=prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "id": "dZ5gPHYW--32"
      },
      "outputs": [],
      "source": [
        "llm_chain_embeddings = HypotheticalDocumentEmbedder(\n",
        "    llm_chain=llm_chain, base_embeddings=base_embeddings\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fj8Dzu1UvHUp"
      },
      "source": [
        "### Loading data from pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UYPVW61oUDS8"
      },
      "source": [
        "Download book from here https://www.gita-society.com/bhagavad-gita-in-english-source-file.pdf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "JpSOGkpTAbpL"
      },
      "outputs": [],
      "source": [
        "# Load the  pdf\n",
        "pdf_folder_path = \"/content/kehe103.pdf\"\n",
        "\n",
        "loader = PyPDFLoader(pdf_folder_path)\n",
        "docs = loader.load_and_split()\n",
        "\n",
        "text_splitter = RecursiveCharacterTextSplitter(\n",
        "    chunk_size=500,\n",
        "    chunk_overlap=50,\n",
        ")\n",
        "documents = text_splitter.split_documents(docs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9x2eA1iRT-G6"
      },
      "source": [
        "### Initialize the Vectorstore"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "result = embeddings.embed_query(\n",
        "    \"What did the president say about Ketanji Brown Jackson\"\n",
        ")"
      ],
      "metadata": {
        "id": "Jb7sw6aDiTzK"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "id": "rXFIs5v7Abt3"
      },
      "outputs": [],
      "source": [
        "from langchain.vectorstores import LanceDB\n",
        "\n",
        "vector_store = LanceDB.from_documents(documents, embeddings)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P4bvTDFDEEX6",
        "outputId": "9a5c2c6d-1bc9-4a7b-83b5-265610dfbae1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Document(metadata={'page': 11, 'source': '/content/kehe103.pdf'}, page_content='Calcium, riboflavin, vitamin A: These essential nutrients are low unless milk or a milkshake is ordered.Folic acid, fibre:  There are few fast food sources of these key factors.\\nFat: The percentage of energy from fat is high in many meal combinations.Sodium: The sodium content of fast food meals is high, which is not desirable.Energy: Common meal combinations contain excessive energy when compared with the amounts of other nutrients provided.'),\n",
              " Document(metadata={'page': 11, 'source': '/content/kehe103.pdf'}, page_content='39population. \\tIf\\tthis\\tis\\tnot\\tmain tained,\\t80\\tper\\tcent\\tof\\tthem\\twill\\tstay\\t \\noverweight \\tas\\tadul ts.\\tThis\\tcan\\tput\\tthem\\tat\\trisk\\tfor\\tmany\\tmedi cal\\tprobl ems,\\t\\nincluding \\tdia betes,\\thigh\\tblo od\\tpre ssure,\\thigh\\tcho lesterol\\tand\\tsle ep\\tapn ea\\t\\n(a\\tsleep\\tdisorder) .\\nTable 2: Nutritional Limitations of Fast Foods\\nThe following factors appear to be the major nutritional limitations of fast-food meals.'),\n",
              " Document(metadata={'page': 11, 'source': '/content/kehe103.pdf'}, page_content='the nutritional needs of teenagers. Both adolescents and health professionals should be aware that fast foods are acceptable nutritionally when they are consumed judiciously and as a part of a well-balanced diet. But when they become the mainstay of the diet there is cause for concern. A nutrient imbalance may not appear to be a problem until a number of years have gone by, unless some specific problem such as a chronic disease exists. However, evidence is accumulating to show that food intake'),\n",
              " Document(metadata={'page': 11, 'source': '/content/kehe103.pdf'}, page_content='Although fast foods can contribute nutrients to the diet, they cannot completely meet')]"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ],
      "source": [
        "# passing in the string query to get some refrence\n",
        "query = (\n",
        "    \"which factors appear to be the major nutritional limitations of fast-food meals\"\n",
        ")\n",
        "\n",
        "vector_store.similarity_search(query)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 127
        },
        "id": "aJ-scBC0qCOx",
        "outputId": "d66ec604-3189-4ac7-fbea-2e081efd3dd8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-68-106fbd6a4823>:1: LangChainDeprecationWarning: The method `Chain.run` was deprecated in langchain 0.1.0 and will be removed in 1.0. Use invoke instead.\n",
            "  llm_chain.run(\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"The major nutritional limitations of fast-food meals can vary depending on the specific meal and location, but some common factors include high levels of saturated fat, sodium, and added sugars. These ingredients can contribute to an increased risk of heart disease, obesity, and other health issues. Additionally, fast-food meals often lack important nutrients such as fiber, vitamins, and minerals. Limited options for fresh fruits and vegetables and the use of processed meats are also common limitations. However, it's important to note that not all fast-food meals are nutritionally limited, and some options may offer healthier choices.\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 68
        }
      ],
      "source": [
        "llm_chain.run(\n",
        "    \"which factors appear to be the major nutritional limitations of fast-food meals\"\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "id": "QTfPxNGhlcN7",
        "outputId": "719a828d-3ace-46c4-d8f3-3bd92d69616a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'\\nPulses and legumes are excellent sources of plant-based protein and dietary fiber. They also contain a variety of essential vitamins and minerals, including iron, potassium, magnesium, and folate. Additionally, they are low in fat and cholesterol, making them a healthy choice for a balanced diet.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 69
        }
      ],
      "source": [
        "llm_chain.run(\"what kind of  Main Nutrients provided by Pulses and Legumes?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        },
        "id": "ObAmmykV-zzs",
        "outputId": "ab11f4b1-5ec3-481b-d5a8-70a41a2711e6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'It is difficult to determine which specific groups serve come from without more context or information. However, it is likely that serve come from a variety of food groups, including grains, vegetables, fruits, dairy, and protein sources. The amount of serve come from each group may vary depending on the specific recipe or dish being served. It is best to consult a nutritionist or refer to a dietary guideline for more specific information on serving sizes and food groups. '"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 70
        }
      ],
      "source": [
        "llm_chain.run(\"serving come from which groups & how much\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C3UvnY6YIKxf"
      },
      "source": [
        "Thanks"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.1"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}